I. Field
The present invention relates generally to multimedia over error-prone networks, and more specifically to video coding and decoding techniques used in conjunction with multimedia over wireless networks.
II. Background
Current video compression techniques, such as, ITU-T H.263, H.263+, H.263++, ISO MPEG-4 and JVT H.264 have become increasingly efficient in compressing video. The ability to achieve good visual quality at relatively low bitrates has led to increased popularity of video applications in emerging multimedia applications over bandwidth limited channels, such as, wireless channels. However, the predictive nature of compression techniques employed by current video codecs makes the compressed video bit-streams very fragile and susceptible to degradation due to packet losses in the channel.
Channel errors or packet loss can lead to loss of synchronization between a video encoder and a decoder. More specifically, a locally decoded copy of a reconstructed video frame at the encoder may not match the corresponding reconstructed video frame at the decoder. This loss of synchronization between a video encoder and a decoder is also sometimes termed “drift” since there will be a drift between the receiver's decoder and the sender's decoder. Drift can lead to significant losses in decoded video quality.
The reduction in decoded quality due to a drift is a direct consequence of the prediction-based coding that underlies current video codecs. That is, to currently decode the current video frame, all previous video frames need to have been reconstructed correctly. This problem is exacerbated in a wireless channel where packet losses are far more frequent than in a wire-line network and tend to occur in bursts.
Re-transmission schemes, such as Automatic Repeat Request (ARQ), or Forward Error Correction (FEC) schemes have been used to combat drift problems between the encoder and decoder and thereby alleviate the loss in quality due to packet losses. However, using ARQ or FEC schemes (or a combination of both) may not prove effective since the latency constraints of the video application may preclude the use of such schemes. Moreover, FEC-based schemes cannot guarantee that the data will be received and if it is not, the drift continues until the next intra-frame1 is received. 1 An intra-frame is used by video encoders to break the prediction loop and hence stop drift.
The Wyner-Ziv Theorem is discussed in A. D. Wyner and J. Ziv, “The rate distortion function for source coding with side information at the decoder,” IEEE Trans. Inf. Theory, vol. 22, pp. 1-10, January 1976. This Theorem addresses the problem of source coding with side-information. Consider two correlated sources X and Y. The encoder needs to compress the source X when only the decoder has access to the source Y. When the mean squared error is the distortion measure and X=Y+N where N has a Gaussian distribution, the rate—distortion performance for coding X is the same whether or not the encoder has access to Y.
Recently, based on the principles of source coding with side-information within a Wyner-Ziv framework joint source-channel coding techniques have been proposed to address the problem of drift. See, for example, R. Puri and K. Ramchandran, “PRISM: A New Robust Video Coding Architecture based on Distributed Compression Principles,” in Allerton Conference on Communication, Control and Computing, 2002; A. Sehgal, A. Jagmohan, and N. Ahuja, “Wyner-ziv Coding of Video: An Error-Resilient Compression Framework,” IEEE Trans. on Multimedia, vol. 6, pp. 249-258, 2004; A. Aaron, S. Rane, R. Zhang, and B. Girod, “Wyner-Ziv Coding of Video: Applications to compression and error resilience,” in Proc. IEEE Data Compression Conf., 2003; and A. Aaron, S. Rane, D. Rebollo-Monedero, and B. Girod, “Systematic Lossy Forward Error Protection for Video Waveforms,” in Proc. IEEE Int. Conf. Image Proc., 2003.
The codecs discussed in R. Puri and K. Ramchandran, “PRISM: A New Robust Video Coding Architecture based on Distributed Compression Principles,” in Allerton Conference on Communication, Control and Computing, 2002; and A. Aaron, S. Rane, R. Zhang, and B. Girod, “Wyner-Ziv Coding of Video: Applications to compression and error resilience,” in Proc. IEEE Data Compression Conf., 2003, are full-fledged video codecs that eschew the predictive coding framework.
On the other hand, the codecs discussed in A. Sehgal, A. Jagmohan, and N. Ahuja, “Wyner-ziv Coding of Video: An Error-Resilient Compression Framework,” IEEE Trans. on Multimedia, vol. 6, pp. 249-258, 2004; and A. Aaron, S. Rane, D. Rebollo-Monedero, and B. Girod, “Systematic Lossy Forward Error Protection for Video Waveforms,” in Proc. IEEE Int. Conf. Image Proc., 2003, retain the predictive coding framework but send some extra information to mitigate the effect of drift.
Notwithstanding these advances, there is a need in the art for techniques that not only enables robust video delivery over wireless channels by helping alleviate the effects of packet losses on compressed video bit-streams transmitted over such wireless channels, but also satisfies the requirements of                Be able to work with existing compression techniques        Have low processing latency        Quickly eliminate/reduce the drift between the video encoder and decoder        Require low bitrate when compared to that used by the video codec        